Skip to content

feat: geofence-based geographic targeting for programs (re-land from #76)#273

Open
gonzalesedwin1123 wants to merge 8 commits into
19.0from
reland/gis-geofence
Open

feat: geofence-based geographic targeting for programs (re-land from #76)#273
gonzalesedwin1123 wants to merge 8 commits into
19.0from
reland/gis-geofence

Conversation

@gonzalesedwin1123

@gonzalesedwin1123 gonzalesedwin1123 commented Jul 2, 2026

Copy link
Copy Markdown
Member

Re-lands the scope described in reverted PR #76 (revert: #271), as part of splitting that PR into description-consistent pieces. This description incorporates the relevant content of #76 so reviewers do not need to open it; everything listed below is contained in THIS PR's diff.

Summary

  • Add spp_program_geofence module for geofence-based program targeting and eligibility management.
  • Extend spp_gis spatial operators to support MultiPolygon and GeometryCollection.
  • Make MapTiler API key optional in GIS UI, with OSM fallback when key is not configured (placeholder key YOUR_MAPTILER_API_KEY_HERE treated as unconfigured).
  • Stabilize GIS edit/map widgets (re-render lifecycle, draw interaction behavior).

Functional detail (from #76, verified against this diff)

spp_program_geofence (new module)

  • Program-level geofence_ids and geofence_count.
  • Geofence eligibility manager (spp.program.membership.manager.geofence) with:
    • Tier 1: coordinate intersection.
    • Tier 2: area intersection fallback (optional, with area type filter).
  • Program configuration UI integration and program creation wizard support.
  • Geofence management views/action/menu, gated by explicit read ACLs for Programs Viewer/Validator roles (menu groups restriction avoids exposing items to users lacking read access).

spp_gis

  • Operator support for complex geometry types via ST_GeomFromGeoJSON; for MultiPolygon/GeometryCollection, (geojson, distance) applies ST_Buffer(...) correctly (EPSG:3857 transform parity when SRID is 4326), with regression tests for the distance-buffer path on both complex types.
  • OSM style fallback in renderer/edit widgets when MapTiler key is missing; placeholder-key handling in the controller.
  • Geofence GeoJSON output includes the record uuid as feature id; new spp.gis.geofence.tag model replaces vocabulary-based geofence tags (+ACLs).

Added on top of #76 (not in the original)

  • Upgrade migration (spp_gis 19.0.2.1.0): geofence tag_ids changes co-model from spp.vocabulary to spp.gis.geofence.tag over the same rel table. Release v19.0.2.0.0 (Biliran) ships the old schema, so a pre-migration parks legacy links (the FK swap would otherwise fail) and a post-migration recreates them as tag records. Fully literal, value-parameterized SQL. Regression tests: remap, dedup across geofences, valid-link survival, fresh-DB no-op.
  • spp_program_geofence: initial readme/HISTORY.md; user-facing strings wrapped in _() for i18n (gemini-code-assist review).
  • spp_gis: readme/DESCRIPTION.md updated to describe geofences, complex-geometry queries, and the OSM fallback; version bump + HISTORY entry; README rendered by CI's pinned generator.

Not in this PR (other pieces of #76, re-landed separately)

spp_hazard CAP severity (#274), spp_cel_domain (#275), spp_api_v2 (#276), PHL demo data (#277), infra/tooling (#278, merged), spp_metric_service (#279), spp_gis_report (#280), spp_api_v2_gis (#281), spp_mis_demo_v2 (#282).

Verification

  • ./spp t spp_gis: 92 passed, 0 failed (includes 4 migration tests)
  • ./spp t spp_program_geofence: 31 passed, 0 failed
  • GHAS odoo-sql-injection-string-format alerts #1837–#1842: fixed (literal SQL)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces geofence-based geographic targeting for OpenSPP programs by adding the new spp_program_geofence module and extending spp_gis to support complex geometry types like MultiPolygon and GeometryCollection. It also replaces vocabulary-based geofence tags with a dedicated spp.gis.geofence.tag model, complete with migration scripts, and implements an OpenStreetMap fallback when no MapTiler API key is configured. The review feedback focuses on improving performance and code quality in the new Odoo models and JS widgets. Key recommendations include avoiding map re-creation on every patch in the GIS edit widget, optimizing recordset operations (using subtraction instead of lambda filtering and avoiding heavy NOT IN clauses in search domains), checking for active records in asynchronous tasks, and wrapping user-facing strings in _() for internationalization.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +224 to +228
def _import_registrants(self, new_beneficiaries, state="draft", do_count=False):
_logger.info("spp_program_geofence: Importing %s beneficiaries", len(new_beneficiaries))
self.env["spp.program.membership"].create(
[{"program_id": self.program_id.id, "partner_id": b.id, "state": state} for b in new_beneficiaries]
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In Odoo, when validating a Many2one relation in asynchronous or background tasks (such as _import_registrants which is queued via delayable), you should explicitly check if the referenced record is active (e.g., not self.program_id.active) in addition to checking its existence. A Many2one field still dereferences a deactivated record as a truthy recordset, which can bypass simple existence checks if the record is deactivated after the task is queued.

Additionally, iterating over new_beneficiaries.ids in the list comprehension is significantly more efficient than iterating over the new_beneficiaries recordset directly, as it avoids instantiating singleton record objects for each iteration.

Suggested change
def _import_registrants(self, new_beneficiaries, state="draft", do_count=False):
_logger.info("spp_program_geofence: Importing %s beneficiaries", len(new_beneficiaries))
self.env["spp.program.membership"].create(
[{"program_id": self.program_id.id, "partner_id": b.id, "state": state} for b in new_beneficiaries]
)
def _import_registrants(self, new_beneficiaries, state="draft", do_count=False):
self.ensure_one()
if not self.program_id or not self.program_id.active:
_logger.warning("spp_program_geofence: Program is inactive or deleted, skipping import.")
return
_logger.info("spp_program_geofence: Importing %s beneficiaries", len(new_beneficiaries))
self.env["spp.program.membership"].create(
[{"program_id": self.program_id.id, "partner_id": partner_id, "state": state} for partner_id in new_beneficiaries.ids]
)
References
  1. In Odoo, when validating a Many2one relation in asynchronous or background tasks, explicitly check if the referenced record is active (e.g., not record.active) in addition to checking its existence. A Many2one field still dereferences a deactivated record as a truthy recordset, which can bypass simple existence checks if the record is deactivated after the task is queued.

Comment on lines 58 to 62
onPatched(() => {
this.defaultZoom = this.map.getZoom();
this.renderMap();
this.onLoadMap();
this.addDrawInteraction();
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Re-creating the entire MapLibre map instance on every patch/update is extremely inefficient. It causes map flickering, resets the map state (such as user zoom/pan), and can rapidly exhaust the browser's WebGL context limit, leading to context loss errors.

Instead, you should check if the geometry value has actually changed, and only re-render the map or update the draw features when a real change occurs. (Note: Remember to also initialize this._lastValue = this.props.record.data[this.props.name]; in onMounted to prevent unnecessary re-renders on the first patch).

        onPatched(() => {
            const currentValue = this.props.record.data[this.props.name];
            if (currentValue !== this._lastValue) {
                this._lastValue = currentValue;
                this.defaultZoom = this.map.getZoom();
                this.renderMap();
                this.addDrawInteraction();
            }
        });

Comment on lines +154 to +157
tier2_domain = base_domain + [
("area_id", "in", matching_areas.ids),
("id", "not in", tier1.ids),
]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using ("id", "not in", tier1.ids) in the search domain is a performance anti-pattern in Odoo when tier1 contains a large number of records. This generates a heavy SQL query with a massive NOT IN clause.

Since the method returns tier1 | tier2 at the end, and Odoo's recordset union operator (|) automatically handles deduplication in Python extremely efficiently, you can safely remove the not in filter from the database query.

                tier2_domain = base_domain + [
                    ("area_id", "in", matching_areas.ids),
                ]

Comment on lines +188 to +189
existing_partner_ids = set(self.program_id.get_beneficiaries().mapped("partner_id").ids)
new_beneficiaries = new_beneficiaries.filtered(lambda r: r.id not in existing_partner_ids)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Filtering a recordset using a Python lambda with not in on a set of IDs is less efficient and less idiomatic than using Odoo's built-in recordset subtraction operator (-).

You can simplify this to a single subtraction operation, which is faster and cleaner.

Suggested change
existing_partner_ids = set(self.program_id.get_beneficiaries().mapped("partner_id").ids)
new_beneficiaries = new_beneficiaries.filtered(lambda r: r.id not in existing_partner_ids)
existing_partners = self.program_id.get_beneficiaries().mapped("partner_id")
new_beneficiaries -= existing_partners

Comment on lines +201 to +202
program.message_post(body=f"Import of {len(new_beneficiaries)} beneficiaries started.")
program.write({"is_locked": True, "locked_reason": "Importing beneficiaries"})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These user-facing log messages and lock reasons are hardcoded strings and should be wrapped in _() to support internationalization (i18n).

Suggested change
program.message_post(body=f"Import of {len(new_beneficiaries)} beneficiaries started.")
program.write({"is_locked": True, "locked_reason": "Importing beneficiaries"})
program.message_post(body=_("Import of %s beneficiaries started.") % len(new_beneficiaries))
program.write({"is_locked": True, "locked_reason": _("Importing beneficiaries")})

Comment on lines +241 to +251
_logger.exception("Geofence eligibility preview failed for manager %s", self.id)
self.preview_count = 0
self.preview_error = "Preview failed. Check the server logs for details."
return {
"type": "ir.actions.client",
"tag": "display_notification",
"params": {
"title": "Preview Complete",
"message": f"{self.preview_count} registrants match the current geofences.",
"sticky": False,
"type": "success" if not self.preview_error else "warning",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These user-facing notification titles, messages, and preview errors are hardcoded strings and should be wrapped in _() to support internationalization (i18n).

            self.preview_error = _("Preview failed. Check the server logs for details.")
        return {
            "type": "ir.actions.client",
            "tag": "display_notification",
            "params": {
                "title": _("Preview Complete"),
                "message": _("%s registrants match the current geofences.") % self.preview_count,
                "sticky": False,
                "type": "success" if not self.preview_error else "warning",
            },
        }

Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.77083% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.59%. Comparing base (bf61488) to head (123c700).
⚠️ Report is 1 commits behind head on 19.0.

Files with missing lines Patch % Lines
spp_program_geofence/models/eligibility_manager.py 77.27% 30 Missing ⚠️
spp_gis/controllers/main.py 0.00% 2 Missing ⚠️
spp_program_geofence/models/program.py 83.33% 2 Missing ⚠️
spp_program_geofence/__manifest__.py 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             19.0     #273      +/-   ##
==========================================
- Coverage   74.86%   71.59%   -3.28%     
==========================================
  Files        1093      282     -811     
  Lines       63718    16587   -47131     
==========================================
- Hits        47701    11875   -35826     
+ Misses      16017     4712   -11305     
Flag Coverage Δ
endpoint_route_handler ?
fastapi ?
spp_aggregation ?
spp_alerts ?
spp_analytics ?
spp_api_v2 ?
spp_api_v2_change_request 66.41% <ø> (ø)
spp_api_v2_cycles 70.65% <ø> (ø)
spp_api_v2_data ?
spp_api_v2_entitlements 69.96% <ø> (ø)
spp_api_v2_gis 71.52% <ø> (ø)
spp_api_v2_products ?
spp_api_v2_programs 92.03% <ø> (ø)
spp_api_v2_service_points ?
spp_api_v2_simulation ?
spp_api_v2_vocabulary ?
spp_approval ?
spp_area ?
spp_area_hdx 81.43% <ø> (ø)
spp_attachment_av_scan ?
spp_audit ?
spp_audit_programs 0.00% <ø> (ø)
spp_banking ?
spp_base_common 90.26% <ø> (ø)
spp_base_setting ?
spp_case_base ?
spp_case_cel ?
spp_case_demo ?
spp_case_entitlements 97.61% <ø> (ø)
spp_case_graduation ?
spp_case_programs 97.14% <ø> (ø)
spp_case_registry ?
spp_case_session ?
spp_cel_domain ?
spp_cel_event ?
spp_cel_registry_search ?
spp_cel_vocabulary ?
spp_change_request_v2 75.46% <ø> (ø)
spp_claim_169 ?
spp_cr_type_assign_program 91.17% <ø> (ø)
spp_cr_types_advanced 0.00% <ø> (ø)
spp_cr_types_base 0.00% <ø> (ø)
spp_dci ?
spp_dci_client ?
spp_dci_client_dr ?
spp_dci_client_ibr ?
spp_dci_client_sr ?
spp_dci_compliance ?
spp_dci_demo 69.23% <ø> (ø)
spp_dci_indicators ?
spp_dci_server ?
spp_dci_server_social ?
spp_demo ?
spp_demo_phl_luzon ?
spp_disability_registry ?
spp_drims ?
spp_drims_sl ?
spp_drims_sl_demo ?
spp_encryption ?
spp_farmer_registry ?
spp_farmer_registry_cr ?
spp_farmer_registry_demo ?
spp_farmer_registry_vocabularies ?
spp_gis 75.30% <92.00%> (+1.20%) ⬆️
spp_gis_indicators ?
spp_gis_report ?
spp_graduation ?
spp_grm ?
spp_grm_case_link ?
spp_grm_demo ?
spp_hazard ?
spp_hazard_programs ?
spp_hxl_area ?
spp_import_match ?
spp_indicator ?
spp_irrigation ?
spp_land_record ?
spp_metric ?
spp_metric_service ?
spp_metrics_core ?
spp_metrics_services ?
spp_mis_demo_v2 ?
spp_oauth ?
spp_program_geofence 80.23% <80.23%> (ø)
spp_programs 65.27% <ø> (ø)
spp_registrant_gis ?
spp_registry 86.83% <ø> (ø)
spp_registry_group_hierarchy ?
spp_scoring ?
spp_scoring_programs ?
spp_security 66.66% <ø> (ø)
spp_service_points ?
spp_simulation ?
spp_starter_disability_registry ?
spp_starter_farmer_registry ?
spp_starter_social_registry ?
spp_starter_sp_mis ?
spp_statistic ?
spp_storage_backend ?
spp_studio ?
spp_studio_change_requests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
spp_gis/__manifest__.py 0.00% <ø> (ø)
spp_gis/models/geofence.py 88.54% <100.00%> (+0.90%) ⬆️
spp_gis/operators.py 81.25% <100.00%> (+7.40%) ⬆️
spp_program_geofence/__init__.py 100.00% <100.00%> (ø)
spp_program_geofence/models/__init__.py 100.00% <100.00%> (ø)
spp_program_geofence/wizard/__init__.py 100.00% <100.00%> (ø)
...p_program_geofence/wizard/create_program_wizard.py 100.00% <100.00%> (ø)
spp_program_geofence/__manifest__.py 0.00% <0.00%> (ø)
spp_gis/controllers/main.py 50.00% <0.00%> (-12.50%) ⬇️
spp_program_geofence/models/program.py 83.33% <83.33%> (ø)
... and 1 more

... and 811 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/post-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
Comment thread spp_gis/migrations/19.0.2.1.0/pre-migration.py Fixed
…tion (from #76)

Re-lands the PR #76 description scope: spp_gis complex-geometry operators
(MultiPolygon/GeometryCollection + distance buffering), OSM fallback when no
MapTiler key is configured, renderer/edit-widget lifecycle fixes, geofence
uuid in GeoJSON output, the spp.gis.geofence.tag model, and the new
spp_program_geofence module (geofence eligibility manager, program UI,
creation wizard).

Adds what the original PR lacked: a migration pair remapping existing
vocabulary-based geofence tag links (release v19.0.2.0.0 schema) onto
spp.gis.geofence.tag records — the rel table is reused, so legacy rows are
parked pre-upgrade and restored post-upgrade. Includes regression tests.
…generate spp_gis README

Semgrep flagged f-string SQL in the tag migration scripts; identifiers were
constants but composed SQL via psycopg2.sql removes the pattern entirely.
README.rst regenerated from the updated HISTORY fragment (in-scope module
only).
Replaces psycopg2.sql identifier composition with prebuilt literal queries
(the aux table name and the valid-link filter are compile-time constants),
clearing the odoo-sql-injection-string-format code-scanning alerts. Values
remain parameterized; behavior unchanged (spp_gis suite 92/0).
Every module carries a readme/HISTORY.md; the module was created in #76
without one. README.rst History section will be aligned to CI's renderer
output in a follow-up commit.
…r-facing strings in _()

- README.rst History section applied verbatim from CI's pre-commit diff.
- Address gemini-code-assist review: translatable message_post/lock-reason and
  preview notification strings (i18n).
…back

DESCRIPTION.md predates the geofence move (#74) and did not mention the
capabilities this PR adds. README will be aligned to CI's renderer output in
a follow-up commit.
@gonzalesedwin1123

Copy link
Copy Markdown
Member Author

Disposition of gemini-code-assist findings:

Applied (commit 9d9afdb): i18n _() wraps for message_post/lock-reason and preview notification strings.

Deferred as follow-ups (pre-existing #76 code; this PR re-lands reviewed behavior and we're keeping the diff scoped):

  • eligibility_manager.py:228 active-record check in the queued import — behavior change needing product decision on deactivated-program semantics.
  • eligibility_manager.py:157 NOT-IN removal and :189 recordset subtraction — valid perf/idiom improvements; tier1/tier2 counts are surfaced separately so the dedup semantics deserve their own reviewed change.
  • field_gis_edit_map.esm.js:62 map re-creation on patch — real concern; needs manual UI verification of zoom/pan/draw state across edits, best done as a dedicated GIS-widget PR.

… ladder

The module's fragments use #/## headings; ### produced an RST title-level
skip that crashed oca-gen-addon-readme.
…fence

READMEs regenerated by CI's pinned generator from the updated DESCRIPTION and
new HISTORY fragments; applied verbatim from its pre-commit diff.
@gonzalesedwin1123

Copy link
Copy Markdown
Member Author

Review & merge order for the PR #76 re-land stack

This PR is part of a 10-PR stack re-landing reverted #76 (revert: #271) in description-consistent pieces. Reviews within a batch can run in parallel; order only matters at merge time. All PR descriptions are self-contained.

Batch 1 — review now, merge in any order (independent, CI green)

PR Scope Review attention points
#273 (this) spp_gis operators/OSM fallback/widgets + new spp_program_geofence Geofence-tag migration (pre/post pair over the reused rel table)
#274 spp_hazard severity→CAP vocabulary + drims/sl_demo/hazard_programs Severity migration mapping: 1→minor, 2→moderate, 3→severe, 4→severe, 5→extreme (deliberate product decision); docs-only commit aligning five base READMEs with CI's renderer
#275 spp_cel_domain: SQL CASE compiler, smart-op lookup, cache tests 27 added edge-path tests (patch coverage 99.3%)
#276 spp_api_v2: OpenAPI polymorphic bodies, OAuth2 auth scheme, bundles Auth middleware change is security-relevant
#277 PHL geodata + spp_demo_phl_luzon + prep script Mostly demo data; batched loader refactor

Batch 2 — merge strictly after prerequisites

PR Merge only after Mechanics
#279 metric-service #275 Auto-retargets to 19.0 on parent merge; wait for its CI to re-run green before merging
#280 gis-report #279 Same retarget-then-CI pattern
#281 api-v2-gis (draft) #273 + #274 + #276 + #279 + #280 (all) Will be rebased onto 19.0 and marked ready-for-review once all five land
#282 mis-demo-v2 (draft) #281 Marked ready once #281 lands

Cautions

Verification summary per PR is in each description; the deferred gemini-code-assist suggestions are documented as comments on the affected PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants